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Abstract: There is a relation between the irreversibihty of thermodynamic pro- 
cesses as expressed by the breaking of time-reversal symmetry, and the entropy 
production in such processes. We explain on an elementary mathematical level the 
relations between entropy production, phase-space contraction and time-reversal 
starting from a deterministic dynamics. Both closed and open systems, in the tran- 
sient and in the steady regime, are considered. The main result identifies under 
general conditions the statistical mechanical entropy production as the source term 
of time-reversal breaking in the path space measure for the evolution of reduced 
variables. This provides a general algorithm for computing the entropy production 
and to understand in a unified way a number of useful (in)cqualities. We also dis- 
cuss the Markov approximation. Important arc a number of old theoretical ideas 
for connecting the microscopic dynamics with thermodynamic behavior. 



1. Introduction 

An essential characteristic of irreversible thermodynamic processes is 
that the time-reversal invariance of the microscopic dynamics is appar- 
ently broken. This means that out of equilibrium a particular sequence 
of macrostates and its time-reversal can have a very different plausibil- 
ity. This, basically, must be the reason for the positivity of transport 
coefficients, or, more generally, for the positivity of entropy produc- 
tion. It has already been argued before in [1^, |T^, mostly via ex- 



amples, how there is a direct relation between entropy production and 
the ratio of probabilities for time-reversed trajectories. Most of this 
was however concentrated on finding a unifying framework for equal- 
ities and inequalities that have recently appeared in nonequilibrium 
statistical mechanics, generalizing, so it is hoped, close to equihbrium 
relations. Most prominent among those is the symmetry expressed in 
the Gallavotti-Cohen fiuctuation theorem, [^,0. In the present pa- 
per, we turn to more fundamental issues for identifying the statistical 
mechanical definition of entropy production rate and to offer a possi- 
ble answer for various interpretational problems that have remained. 
The emphasis is on the simplicity of the explanation avoiding technical 
issues. 
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2. Results 

Nonequilibrium statistical mechanics is to a large extent still under 
construction. Recently, there have been made various proposals for a 
definition of statistical mechanical entropy production going beyond 
the close to equilibrium regime and through which fiuctuations in ir- 
reversible processes could be studied. In some cases, the theory of 
dynamical systems has been a source of inspiration and it was argued 
that phase space contraction can be identified with entropy production 
with nonequilibrium ensembles obtained as limits of ergodic averages, 
see e.g. [0, A somewhat different approach started from taking 
advantage of the Gibbsian structure of the distribution of space-time 
histories where entropy production appeared as the source term for 
time-reversal breaking in the space-time action functional. These two 
approaches have in fact much in common, at least concerning the math- 



ematical analysis, see . Since then, many examples have passed the 
test of verifying that the various algorithms indeed give rise to the 
physical entropy production. There has however not been a derivation 
from first principles to convince also the stubborn that the algorithm 
of |jT6|, |T^ applied to models in nonequilibrium dynamics to identify 



the entropy production, is entirely trustworthy. The main result of the 
present paper is to give such a derivation: that indeed under very gen- 
eral conditions, both for closed systems and for open systems, both in 
the transient regime and in the steady state regime, the entropy pro- 
duction can be obtained as the source term of time-reversal breaking 
in the action functional of the path space measure that gives the distri- 
bution of the histories (on some thermodynmaic scale) of the system. 
This representation is useful because it gives the entropy production 
as a function of the trajectories and it allows easy mathematical ma- 
nipulations for taking the average (to prove that it is positive) and 
for studying the fiuctuations (to understand symmetries under time- 
reversal). 

This paper is more or less self-contained with a first Section ^ in- 
troducing the main actors. Sections ^ and ^ contain the main result. 
The difference is that | is entirely about the transient regime for closed 
systems, while Section | deals with open systems and discusses the 
steady state regime. Sections ^ and ^ discuss their consequences in 
the Markov approximation. Section ^ relates the approach to results 
inspired by the theory of chaotic dynamical systems, in particular how 
phase space contraction can play the role of entropy production. Along 
the way, we suggest interpretations that we think are helpful for start- 
ing nonequilibrium statistical mechanics. 
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3. Set-up 

3.1. Phase space and microscopic dynamics. Let Q be the phase 
space of a closed isolated mechanical system with x ^ Q representing 
the microstates, i.e., as described by canonical variables for a classical 
system of particles, x = (gi, . . . , qN,Pi, ■ ■ ■ , Qn)- The Hamiltonian 
dynamics specifies the flow x ^ <Pt{x) on VL under which x (at some 
initial time to) evolves into at time to + The dynamics is 

reversible in the sense that 7r</)t7r = ^^""^ where the time-reversal vr on 
Vt is the involution that changes the sign of the momenta pj. The flow 
preserves the phase space volume (Liouville's theorem); the Jacobian 
determinant equals unity, \d(l)t{x) / dx\ = 1 for each t and the Liouville 
measure dx is time-invariant. 

We fix a time- interval 6 and write / = 0^. Of course, / preserves the 
phase space volume and tt/tt = f~^. 



3.2. Reduced variables. The time evolution preserves the total en- 
ergy. We introduce therefore the state space Qe = T, the energy shell, 
corresponding to a fixed total energy E or better, some interval around 
it. We denote by |y4| the phase space volume of a region A G T given by 
the projection p of the Liouville measure into F. Since F is thought of 
as containing a huge number of degrees of freedom, it is reasonable to 
divide it further. For comparison with experimental situations, we look 
at some special set of variables, suitably defined in each case, which 
give a more coarse-grained, contracted, or reduced description of the 
system, fl^, |l3l. Depending on the context or on the application. 



their precise nature may vary. It could be that we look at macroscopic 
variables a{x) implying a subdivision of F by cutting it up in phase 
cells defined by a < a{x) < a + Aa (with some tolerance Aa), or that 
we split up X = {y,z) into an observable part y and a background 
part z. For example, the y might refer to the coordinates of the par- 
ticles in a subsystem while the background is only monitored as the 
macrostate of reservoir (s). 

At this moment, we do not commit ourselves to one picture but rather 
imagine somewhat abstractly a map M : T ^ T : x ^ M{x) where 
F is the reduced phase space, a finite partition of F. When having in 
mind macrostates, this space F would correspond to the /i— space of 
Gibbs. The fact that this partition is assumed finite is not realistic, 
it is more like Mf^, but it is convenient for the notation and it is not 
essential. With some abuse of notation, the elements of F are denoted 
by M (standing for all possible values of M{x)) and of course, every 
microscopic trajectory 7 = (x, fx, . . . , f^x) gives rise to a trajectory 
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uj = {M{x), M{fx), . . . , M{f"x)) in F. We also assume for simplicity 
that ttM is well defined via ttM = Mtt, that is Mx = My implies 
Mirx = Mny, for all x, ?/ G F. 



3.3. Distributions. Probabilities enter the description because the 
exact microstate of the system is not accessible to us. This is so when 
preparing the system and also later when we observe the system. Even 
when we know the reduced state, we still need to evaluate the plausibil- 
ity of background configurations in order to predict the future develop- 
ment on the level of the reduced states. A natural choice here is to use 
the microcanonical ensemble. That is, we sample the reduced variables 
according to some probability distribution z/ on F and we impose the 
microcanonical distribution on each phase cell M. If i> is a probability 
on F, then x p[x) = C'{M{x))/\M{x)\ is the probability density on F 
obtained from u by uniform randomization (microcanonical ensemble) 
inside each M G f . It is uniquely determined from the two conditions 
(1) z> X p(M) = z>(M) and (2) z> x p{x\M) = 1/|M|, x e M, M et. 
(Remark: the writing x p has no meaning in itself except that it is 
the notation we use for this probability density.) In words, the proba- 
bility of a microstate x is the probability (under 0) of its corresponding 
reduced state Mx multiplied with the a priori probability (under the 
Liouville measure) of x given the reduced state Mx. So if we take 
z> = 6{M — •) concentrated on the reduced state M G F, then x p 
is the initial probability density corresponding to an experiment where 
the system is started in equilibrium subject to constraints; that is a 
uniform (i.e., microcanonical) distribution of the phase points over the 
set M. 

For the opposite direction, we note that every density z/ on F gives rise 
to its projection J9(z/), a probability on F, via 

p(z/)(M) = v{M) = J dxv{x)5{M{x) - M) 

and obviously, p(z/ x p) = v. All this is very much like what enters in 
projection-operator techniques, p3| . 



It now makes sense to ask for the probabilities on F at time t, given 
that the system started at time zero in Mq G F; we always mean by 
this that the microstates were uniformly sampled out of Mq. They are 
given by the ratio 

VmhiUx) e A I M{x) = Mo] = ''^^ ',^1^1^°' (3-1) 

\Mn\ 
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More generally, when, for all we know at time zero, the statistics of the 
reduced variables is given in terms of the probability z> on F, then, at 
time t, the statistics on F is obtained from 

ut =p{{C'X p)t) 

where (z>x p)t gives the distribution at time t as solution of the Liouville 
equation with initial distribution P x p on F. 

Finally, given an initial probability z/ on F, we can look at the collection 
of all paths uj = (M(x), M(/x), . . . , M{f'^x)), {n fixed) where, first, 
a reduced state Mq is drawn according to and then, with uniform 
probability on Mq a microstate x G Mq is drawn (so that M{x) = Mq). 
We denote the resulting distribution on these paths by P^; it defines 
the path space measure on trajectories in F. 

3.4. Entropies. There will be various types of entropies appearing in 
what follows, each having their specific role and meaning. There is first 
the Shannon entropy S'(/i), a functional on probability densities on 
F: 

S{n) = — J dxjj,{x)\njj,{x) (3.2) 

We can also define the Shannon entropy S{fi) for probability laws on 
F through 

S{fl) = -J2KM)\nfl{M) (3.3) 

M 

There is secondly the Boltzmann entropy Sb which is first defined on 
M G F, and then for each microstate x G F as 

SsiM) = In |M|; Ssix) = ^b(M(x)) (3.4) 

The dependence on the number of particles is ignored here as it shall 
be of no concern. Most frequently, we have in mind here macroscopic 
variables (such as density and/or velocity profile(s)) for characterizing 
the reduced states. Any two microstates on F are a priori equivalent 
but if we randomly pick a microstate x from F, the chance that its 
reduced state M{x) equals M G F increases with greater Boltzmann 
entropy Sb{M). We can then expect, both for the forward evolution 
and for the backward evolution (positive or negative times) that the 
Boltzmann entropy should increase. This time-reflection invariance 
of the increase of entropy is an instance of the dynamic reversibility 
and it interprets the paradoxical words of Boltzmann when speaking 
about the increase of entropy (minus the H— functional) "that every 



point of the iJ— curve is a maximum," see |T^. From this, equilibrium 
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is understood as the state of maximal entropy, given the constraints 
in terms of macroscopic values that define the equilibrium conditions 
(such as energy, volume and number of particles). Upon varying the 
constraints, this maximal Boltzmann entropy will behave as the ther- 
modynamic entropy (defined operationally). Yet, even out of equilib- 
rium the Boltzmann entropy makes sense which is essential and needed 
even to discuss fluctuations around equilibrium. It then continues to 
correspond to the thermodynamic entropy in the close to equilibrium 
treatments of irreversible processes. 

The Boltzmann entropy thus tells how typical a macroscopic appear- 
ance is from counting its possible microscopic realizations. Also the 
Shannon entropy has its origin in counting (for example in evaluat- 
ing Stirling's formula or other combinatorial computations) and it is 
therefore not surprising that there are relations between the two. For 
our context, the following identity between Shannon and Boltzmann 
entropies holds: 

S{i> X p) - S{i)) = J2 '>iM)SB{M) (3.5) 

M 

Thirdly, we will need the Gibbs entropy S'g'(z/) which is a functional on 
the statistics i> of reduced states: 

5'g'(z>) = sup S{n) (3.6) 
p{^l)=^> 

Equivalent ly, 

Sg{i>) = S{i) X p) (3.7) 

because we always have p{C> x p) = D and a standard computation 
shows that S'(z/ x p) > S{p) for every density p on F for which also 
p(p) = (Gibbs variational principle). At the same time, from ( p.5|) , 
note that for = 6{M — ■), 

Sb(M) = S{i) X p) 

Combining this with ( pj.T] ), we observe that in case z> concentrates on 
one M G fi, then the Boltzmann and the Gibbs entropies coincide: 

Sb{M) = Sg{S{M - ■)) (3.8) 

which indicates that the Gibbs entropy is, mathematically speaking, 
an extension of the Boltzmann entropy since it lifts the definition of 
Sb to the level of distributions on F. 

Another application of the Gibbs variational formula (|3.6| ) concerns 
the change of entropy under the Hamiltonian dynamics. This was em- 
phasized by Jaynes, see e.g. Suppose the system is initially (at 
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time to = 0) prepared with density x p (i.e., microcanonically with 
reduced states sampled from z>). Then, according to the Liouville equa- 
tion, at time t we obtain the density (z>x p)^. But only the reduced state 
is monitored, i.e., its projection p((z> x p)t) = Of. This is for example 
obtained from the empirical distribution of the macrovariables. From 
( |3.6| ) it follows that SciC't) > 'S'Gr(^') because, by Liouville's theorem, 
S{{0 X p)t) = S{i> X p) = Sg{i>)- We call the difference 

Scih) — Sg{v) = the (Gibbs-) entropy production 

It is always non-negative. From ( ^.81 ), if we initially prepare the sys- 
tem in some specific reduced state Mq G T, then this (Gibbs-) entropy 
production equals, in fact, Scipt) — Sb{x) > 0,x G Mq. If the set of 
reduced variables allows a hydrodynamic description in which, repro- 
ducibly for almost all x G Mq, M{(j)tx) = Mf G F, then the experi- 
mentalist will, for all practical purposes, identify z>f with 6{Mt — ■) and 
the (Gibbs-) entropy production is then given by the change Sb^Mi) — 
Sb{Mq) = SB{(ptx) — Sb{x) in Boltzmann entropy. In other words, 
the (Gibbs-) entropy production then coincides with the (Boltzmann-) 
entropy production. Yet, from this we see that, under second law 
conditions, the inequality Scii^t) > Scii') obtained from the Gibbs 
variational principle is doing great injustice to the actual difference be- 
tween initial and final Boltzmann entropies: the value of Sb{Mq) will be 
very small compared to the equilibrium entropy In (^ SbIMi) for 
I Mil ~ |F|) if Mq corresponds to a preparation in a special nonequilib- 
rium state. As it is often correctly emphasized, the stone-wall character 
of the second law derives from the great discrepancy in microscopic and 
macroscopic scales as a result of the huge number of degrees of free- 
dom in a thermodynamic system. Moreover, a theoretical advantage 
of considering SB{4>tx) — Sb{x) is that this is directly defined on the 
phase space F and in fact allows a microscopic derivation of the sec- 
ond law based on statistical considerations concerning the initial state, 
see [1^ for recent discussions. Note however that this advantage 
also implies the challenge to relate the Boltzmann entropy with more 
operational definitions of entropy as practiced in thermodynamics of 
irreversible processes where entropy production appears as the prod- 
ucts of fluxes and forces, as obtained from entropy balance equations. 
Yet, irreversible thermodynamics is restricted by the assumption of lo- 
cal equilibrium whose validity requires systems close to equilibrium. 
Finally, there is the dynamical entropy Sk that is an immediate exten- 
sion of the Boltzmann entropy but defined on trajectories: for a given 
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trajectory oj = (Mq, Mi, . . . , M„) in f , we put 

SK{uj) = in\n]^,rm,\ (3.9) 

counting the microstates x G F for which M{f^x) = Mj{= uj), j = 
0, . . . ,n. In ergodic theory, this dynamical entropy ( ^.91 ) is related to 



the Kolmogorov-Sinai entropy via Breiman's theorem, . In this way, 
the Kolmogorov-Sinai entropy gives the asymptotics of the number of 
different types of trajectories as time tends to infinity. Note however 
that in the physical case we have in mind, T does not correspond to 
some kind of mathematical coarse graining and there is no way in which 
it is assumed generating nor do we intend to let it shrink F ^ F. 

3.5. Transient versus steady state regime. The family of nonequi- 
librium states is much more rich and varied than what is encountered 
in equilibrium. It is often instructive to divide nonequilibrium phenom- 
ena according to their appearance in the transient versus the steady 
state regime. The simplest example of a transient regime is when the 
total system starts from a nonequilibrium state and is allowed to re- 
lax to equilibrium. Steady state on the other hand refers to a main- 
tained nonequilibrium state. For this we need an open system that 
is driven away from equilibrium by an environment. All of this how- 
ever strongly depends on the type of variables that are considered and 
over what length and time scales. In this paper we take the point of 
view that the steady state is a special or limiting case of the tran- 
sient situation. Since the fundamental dynamics is Hamiltonian for a 
closed system, any attempt to give a microscopic definition of entropy 
production must start there. We can then discuss the limiting cases 
or approximations through which we are able to define also the mean 
entropy production rate in the steady state. In any case, the identifi- 
cation of the statistical mechanical entropy production as function of 
the trajectory must be independent from the regime, be it transient or 
steady. 

4. Entropy production: closed systems 

Our main goal in the following two sections is to show how, via 
time-reversal, we can define a function on path space which will be rec- 
ognized as the variable or statistical mechanical entropy production. It 
enables us to compute the time-derivative of the entropy production, 
the entropy production rate. Since this function is defined on trajec- 
tories, we can also study its fluctuations. 

The situation to have in mind here is that of the transient regime in a 
closed system; entropy production is the change of entropy. 
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Recall We start by observing that for any two reduced states 

Mq, Mn € r and for every microscopic trajectory 7 corresponding to a 
sequence of microstates starting in Mq and ending in Mn, 

Prob[(x,/x,... =7|M(x) =Mo] 



Prob[(x, fx,... , px) = 76 I M{x) = 7rM„] 

Sb{M^) - Sb{Mo) (4.1) 

where 76 is the time-reversed microscopic trajectory. That 76 is also 
a microscopic trajectory is an expression of dynamic reversibility. This 
identity ( ^.l] ) follows because, given the initial reduced state, the prob- 
ability that a specific microscopic trajectory is realized only depends 
on the probability of the initial microstate. But since we know to what 
reduced state it belongs, that probability is just the exponential of 
minus the Boltzmann entropy. 

While the previous relation indicates that time-reversal transforma- 
tions are able to pick up the entropy production, we cannot in practice 
sample microscopic trajectories. In order to lift these relations to the 
level of trajectories on T, we should also relax the condition that we 
start in a fixed reduced state; if we only know the reduced state the 
dynamics started from, we will not know in what specific reduced state 
we land after time t. For this additional uncertainty, there is a small 
price to be paid. 

Let uj = (Mo, Ml,... ,M„) be a possible trajectory on T. Its time- 
reversal is uQ = (vrM„, . . . ,7rMo). Let fi and i> be two probabilities 
on r. We ask for the ratio of probabilities that the actual trajectory 
coincides with uj and with ujQ, conditioned on starting the microscopic 
trajectory sampled from fi x p and from On x p respectively: 



Prob^xp [trajectory = u] _ p{Mo) |M„ 



Prob,)7rxp [trajectory = uj&] ^{Mn) |Mo| 



(4.2) 



More precisely, this wants to say that the corresponding path space 
measures have a density with respect to each other, given by 

dPf. _ KMq) |M„| 



dP,^e' ' z>(M„)|Mo 
To prove this, it suffices to see that, on the one hand 

p X p (n^=o/-^M,) = ^1 n^^o f-'M, 
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and, on the other hand, for the denominator in the left-hand side of 



where we first used tt/^^tt = / and then the stationarity of the Li- 
ouville measure under /. Hence, the factor | n"^Q f~^Mj\ will cancel 
when taking the ratio as in (4.2)- (|4.3| ); this expresses the time-reversal 



invariance of the dynamical entropy, Sxi^^) = SxiQ^^), which excludes 
it as candidate for entropy production. 

The most interesting case is obtained by taking v = fit for t = n6 



in (|4.2|) or (|4.3| ). Remember that fit = x p)t) is the projection 
on the reduced states of the measure at time t when started from the 
constrained equilibrium fi x p. We then get as a direct consequence of 



Proposition 4.1. For every probability fi on T , 
dP- 

l^TB-n^M = [Sb{M - Sb{x)] + [- In /ii(M(0ix)) + In /i(Mx)] 

(4.4) 

for all trajectories uo = (M (x), M{fx), . . . , M{f^x)) in T, x & T, t = 
nS. 



The right-hand side of ( ^.4| ) contains two contributions. The first 



difference of Boltzmann entropies has already appeared (alone) in ( [4.1| ) 
when the comparison was made between probabilities for microscopic 
trajectories. The second contribution to ( ^^ can thus be viewed as 
originating from the 'stochasticity' of the reduced dynamics. Note in 
particular that even when fi is concentrated on some M G F, then 
fit is still smeared out over various possible reduced states. Yet, this 
second contribution can be expected to be very small under second 
law conditions. After all, if fi{M) = 5{M - M{x)), then p{Mx) = 1. 
And if M{f'^x) is large in the sense that 'typically' almost all z G 
r get into M{f^x) after a sufficient time t = n6, then, we expect, 
\M{(j)tx) n (f)tM{x)\ ~ |M(x)| so that by (§]Tp, also fit{M{(ptx)) ~ 1 
and hence only the first Boltzmann contribution survives. Of course, 
for smaller systems, there is hardly a notion of what is typical but we 
can see what to expect in general. 

Let stand for the expectation with respect to the path space 
measure P^. 

Proposition 4.2. Denote ( \4-4\) by 



Rl = In ■ 
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Then, 

E^le-'^l] = 1 (4.5) 
In particular, its expectation equals the (Gibbs-) entropy production: 
EM]=SGiM-SGifi)>Q (4.6) 



Proof. The identity (|4.5| ) is the normahzation 

TP \dPjHT^l -, 



The relation ([4.6|) follows from (p.5|) and the definition (p.6|)-( pr7D of 



Gibbs entropy from inspecting the expectation of the right-hand side 
of ( |4.4| ). The non- negativity of the (Gibbs-) entropy production was 
already explained under Section p.4| but it can also be obtained from 
applying the Jensen (convexity) inequality to ( [4.5| ). □ 

Remark 1: The above calculations have relied heavily on the structure 
/t X p of the distributions. We will see in Section |] what happens in 
the case where the distribution has the form fl x u where ft will refer 
to the distribution of a subsystem and the u takes into account the 
macrostate of the enviroment that further constraints the evolution. 
Remark 2: One may wonder about the physical significance of the 
terms [— In fit{M {(ptx)) + \iafi{Mx)] appearing in ( [4.4|) . There is no 
general answer: they have a priori nothing to do with entropy produc- 
tion but their addition can become physically significant to the same 
extent that /t and fit are physically motivated. Nevertheless we con- 
tinue to call the right-hand side of 

[-ln/ij(u;„) + \nf,{LJo)] + [^'^(t^n) - Ssiujo)] = R\{u;) 

the total variable (or statistical mechanical) entropy production i?^. 
It coincides with the (Boltzmann-) entropy production under second 
law conditions and its expectation is the (Gibbs-) entropy production. 
Even though i?^ does not quite coincide with the change of Boltzmann 
entropy, it has a useful structure (as ratio of two probabilities) making 
the studies of its fluctuations much easier, see for example ( |4.5| ). The 
amazing point is that while Pfi{uj) and Pp_^T^{Quj) both depend on the 
entire path u = (ci;o,co'i, . . . ,c<j„), their ratio is a state function, only 
depending on the initial and final states ujq and Un- We will use it 
throughout the following. 
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5. Entropy production: open systems 

As a general remark, in the set-up of the present paper, nonequihb- 
rium steady states correspond in reahty to a transient regime for the 
whole system over timescales short enough for the macrostates of the 
reservoirs not to have changed appreciably while long enough for the 
internal subsystem to have reached a stationary condition. We start 
however again from the Hamiltonian dynamics of the total system. 

We consider the situation where the complete system consists of one 
smaller and one larger subsystem. The latter represents an environment 
to which the smaller subsystem is coupled and it can have the form of 
one or more thermal reservoirs, for instance. For short, the smaller 
subsystem will simply be called system. The total phase space is f2 = 
X where the superscripts and 1 stand for the system and for 
the environment, respectively. The dynamics of the total system is 
Hamiltonian and we use the same notation as introduced in Section 
namely / is the discretized flow and the time-reversal vr on f2 is assumed 
to have the form n = ® (we will soon forget these superscripts). 
The reduced picture is given by the partition Cl of Q, with the product 
structure f2 = x fi^. One choice could be taking Cl^ = in case 
the system has only a few microscopic degrees of freedom, and the 
elements of Cl^ corresponding to fixed values of the energies of the 
individual reservoirs. Preparing the system in the initial state /i° and, 
independently, preparing the environment in /t^, we construct the initial 
distribution on Q, from which the microstates are sampled, as (S> 
(x^)x p. At this moment the microscopic dynamics, conserving the total 
energy, takes over and the system gets coupled with the environment. 
We then get, as used in the previous section and as defined in Section 



Ol , a path space measure Pp_o^p_i for the trajectories. 

It is convenient to rephrase the above construction in the following, 
more formal, way to make a connection with the scenario of the previ- 
ous section. We introduce yet another coarse-graining in which only the 
system is observed, while the macrostate of the environment is ignored. 
It is defined via the map p : Cl Cl^ which assigns to every (M, E) & Cl 
its first coordinate M G That means that we actually deal with 
two successive partitions: is a partition of the original phase space fl 
(involving both system and environment) and Q is taken as a further 
partition of Cl. Cl can be identified with Cl^ involving only the system's 
degrees of freedom. The elements of Cl are written as M and those of Q 
as (M, E). To every {y, z) & Q = xQ^ we thus associate an element 
My e Cl and an element {My, Ez) G Vt. 



13 



On Q we put the distribution fi = fi^ which stands for the initial sta- 
tistics of the small subsystem. On Q we put the distribution fi for 
which we only ask that fi{M,E) = fi{M)fi^{E) thus representing the 
preparation of the environment. The first and crucial observation we 
make is that 

{fi^ ^ fi^) X p = ft X {fi X p) (5.1) 
The right-hand side is defined as the generalization of the randomiza- 



tion we introduced in Section |3.3| : given a distribution z/ on and a 
distribution /i on Q we let 

Jixu{y,z) = -p{My)^^'^'''^ 



u{My) 



Take here u = fixp, then z/(?/, z) = fi{My, Ez)/\My\\Ez\ and v{My) 
fi{My). Therefore, 

^ , ^ fi'^(My)fi\Ez) 



\My\\Ez\ 

which proves the identity (|5.1| ). 

The representation ( |5.1| ) enables us to consider the initial distribution 
as constructed directly from the p by randomizing it with the a priori 
distribution px p which is not time-invariant under the dynamics and 
depends on the initial state of the environment, cf. the first remark at 
the end of Section ^. The probability of a trajectory u) = {ujo, ... , cUn) 
in Q, i.e., of the system, may then be evaluated as follows: 



pipol^ 
p{uJo) 

m 



p^o^^i(c.) ^{px{px p))in]^j-^u,) = X p){n]^j-^u,) 



(5.2) 



where P[i{Cj) is the probability of the trajectory u) on started from 
p: 

P^icu)^ipxp)in]^J-^u;,) 

and we sum in ( |5.2|) over all trajectories on Cl (i.e., for environment 
and system together) that coincide in their first coordinate with the 
given trajectory u. Similarly, for the time-reversed trajectory Qu with 
the system's initial distribution z/vr we have the probability (the initial 
microstates sampled from the distribution z/vr x (pir x p) = (z/°7r ® 
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/iV) X p) 



UJ) 



n) \UJo\ 



(5.3) 



where we used a version of (|4.3|). As always, we want to take the ratio 



K^n) .f-f. K^o) \<^' ' ^ 



of ( ^.21) and (|5.3| ). We write this out in the most exphcit form: 



where 

fiHEu) \Mo\\Eo 



EEo,...,E„Pf^[iMo,Eo),... ,{Mn,En)] 

for a trajectory lj = (Mq, . . . , M„) of the system and the sums are over 
trajectories {Eq, . . . , En) of the environment. 

The identity (|5.4|) is still exact and general. To proceed, we choose 
first to be more specific about the nature of the environment. As exam- 
ple, we suppose that the environment consists of m heat baths which 
are taken very large and which are prepared at inverse temperatures 
Pi, . . . , Pm- This means that Cl^ and fi^ are split further as m— fold 
products and that the trajectories of the environment are obtained from 
the successive values of the energies E^ at times i6 in all heat baths, 
k = 1, . . . ,m. We also suppose that these reservoirs are spatially sep- 
arated, each being in direct contact only with the system. Through 
these system-heat bath interfaces a heat current will fiow changing the 
energy contents of each reservoir. It implies that even though the ini- 
tial energies Eq are sampled from fi^ giving inverse temperatures Pk to 
each of the heat baths, a priori it need not be that the final energies E^ 
can be considered as sampled corresponding to the same temperatures. 
At this moment we need a steady state assumption for the reservoirs: 
that they maintain the same temperature during the evolution, or more 
precisely, 

Al: The path space measure P/i gives full measure to those trajectories 
for which 

\E'n - E'o\ < oiW) 

of the order of less than the square root of the size of the volume V of 
the environment. 
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For simplicity, we have used here o{\'V) as a safe estimate. One 
could assume much less: the energy differences \E^ — Eq \ will be of the 
order of the product of the surface area dkA through which heat will 
flow between the k—th heat bath and the system, and the heat current 
(i.e., the flow of energy per unit area and per unit time) and the time 
t = n6. One can recognize this from the calculations in Appendix B. 
We need this assumption Al to get rid of the ratio /i^(i?„)//i^(£'o) 
in (O). Under Al, this ratio is essentially equal to one, when the 



initial dispersion of the energy values under fi^ is much larger than 
the changes of energy. For simplicity (but without loss of generality) 
we can suppose that for each heat bath of spatial size V, fi^ gives the 
uniform distribution over an interval [£'^ — e,S^ + e] where we take 
S'^ = 0{V) and e = o{V). This is just the simplest representation for 
an initial distribution that is peaked at energy value S with deviations 
of order e = \fV say. Within such an interval, the temperatures 
are essentially constant if the size of the interval is large compared with 
the possible changes of the energy due to the flows from the system. 
These statements become sharp in the limit V ^ oo but for finite 
reservoirs this steady state assumption can be expected reahzed over 
times t < t* with t* = t*(V) growing with V less than as \/V. As 
conclusion, we set fi^ (En) / fi^ (Eq) = 1 in 
For notation, we denote by 



4(M„)-4(Mo)=lni^ (5.5) 



|Mo| 



the change of Boltzmann entropy of the system. Usually, at least in 
close to equilibrium treatments, this change is divided into two parts: 
one corresponding to the entropy production properly speaking and one 
term corresponding to the entropy current through the surface of the 
system; one refers to it as an entropy balance equation, see Appendix 
D for a short discussion. The latter, the entropy current, is responsible 
for the change of entropy in the reservoirs, here 



\F I 

^^ T^ = Y.l^BiE'j - SUE',)] (5.6) 



k=l 



the sum of changes of the Boltzmann entropy in each bath. The sum of 
(|575| ) and ( |5.6| ) is the total change of entropy (where total refers to the 
closed system consisting of the (smaller) system and all the reservoirs) 
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and thus equals the entropy production. We write it as 

AS Bio) = SsiMn) - SsiMo) (5.7) 



— m ■ 



As in (|4.4|), it is natural to take = fi^ = p(fi x (/i x p))t in 



which corresponds to the projection at time t = n5 on the system, and 
we obtain a first 

Analogue of Proposition |4.1j: 



P,o^,.(g ^ ^ [-ln/i°(u;„) + ln/x°(u;o)] (5.8) 



We can still do better by reconsidering ( |5.7|) , in particular the ex- 
pectation over the path-space measure of the exponential change of 
Boltzmann entropies in the heat baths. These changes are caused by 
the heat dissipated in each of the reservoirs and it therefore corresponds 
to the entropy current. Since this is the energy current divided by the 
temperature of the reservoir, it should be possible to express it directly 
in terms of the microscopic trajectory over the surface separating the 
heat bath from the system. The basic question is now in what sense 
the trajectory uj of the system determines the trajectory u of the total 
system. In the context of Hamiltonian dynamics it is not hard to see 
that (again, provided that the reservoirs are coupled to different parts 
of the system) the trajectory a) is uniquely determined by its projection 
onto the system, lj, and by the initial energies of the reservoirs, E^. 
We formulate this in the form of another assumption: 



A2: Let a) and u' be two trajectories of the total system such that 
p{(2}) = p{ijj') = UJ. Then, for typical trajectories obtained as succes- 
sive reduced states from the Hamiltonian dynamics, the energy changes 
E^ — Eq depend only on lj. 

This should be understood in the sense of all allowed trajectories, 
typical here referring to the path-space measure P^. This assumption 
is too strong. The problem is that we also consider reduced states on 
the level of the system itself and that the Co is not the continuous time 
microscopic trajectory of the system. It would for example have been 
better to use a finer time-scale for the evolution of the reduced states of 
the system (compared with that of the reservoirs). One could remedy 
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that but we prefer to stick to A2 for simplicity, see Appendix B. 
Assuming A2, we already know that AE^ = — only depends 
on ui : AE'^ = AE^{ui). So, the change of Boltzmann entropy in each 
reservoir, ^|(^^) - 5^(^o) = S^Eq + AE^{u:)) - S'^{e{), depends 
on Lj and Eq. In order to get rid of the dependence on the initial state 
of the reservoirs, we must again use the distribution fi^ and that the 
reservoirs are large. After all, thermodynamic behavior implies that 
the = 0{V) and S'^{Ef) = 0{V), all of the order of the volume V 
of the reservoirs, while Pk = dS^{Ef) / dE^ is kept fixed. This is again 
our steady state assumption Al for the environment; the reservoirs are 
heat baths at a fixed temperature. The change of Boltzmann entropy 
in each of the reservoirs is then 



V 

for all trajectories of the system u and essentially all initial energies 
Eq G {S^ — e, S^+e) (most of trajectories started inside the interval will 
not leave it). This implies that the total change of entropy appearing 
in ( [5.7| ) and in ( |5.(j| ) is, in good approximation. 



ASb{oo) = SU^n) - SU^o) + J2f^kAE\u) (5.9) 



What we have gained with respect to (|5.7|) - (|5.8| ) is that this variable 
entropy production only depends on jl^ through the initial tempera- 
tures of the reservoirs. 
We denote 

We conclude with the final result obtained under the assumptions 
above: 

Analogue of Proposition |4.1j : 

i?J^*(cu) = S° (cu„) - SliiDo) + Y,PkAE\uj) + [-ln/i°(cu„) + ln/i°(cuo)] 

k 

(5.11) 



There are two big modifications with respect to (|4.4| ) for closed sys- 
tems. First, it is important to remember that the AE'^{ijj) in ( |5.11|) 
are in general not differences of the form E^{ujn) — Eq{uq) but they 
represent the heat flow depending on the complete path u. So the 
right-hand side of ( p.ll| ) is not a difference. Secondly, here it is very 
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well possible to take /i° = fi^ at least for small enough times t (com- 
pared to VV) what refers to the full steady state. In other words, we 
can study the stationary regime where the distribution /i° of the system 
is time-invariant. 

Remark 1: The same construction applies of course also when the 
system is coupled to only one reservoir or to various reservoirs at the 
same temperature We take Cl^ = = Q so that the system is 
described via its microscopic states y. The trajectory uj gives successive 
microscopic states lj = {yo, . . . , and the first two terms on the right- 
hand side of (|5.11 ) are identically zero. By energy conservation, the 
total change of energy Ylki-^n ~ ^o) ~ reservoirs is always 

of the form /\E = H{yo) — H{yn), the difference of the initial and final 
energies of the system. Therefore, in (|5.9| ), 

ASb{oo) = P[H{yo) - H{yr.)] 

It is interesting to see that then, when taking fi^{y) = fitiy) ~ exp[— 
a Gibbs measure at inverse temperature /?, the expression (|5.11|) be- 
comes zero. 

Remark 2: The same construction applies also to other scenario's (in- 
stead of via heat reservoirs) but it needs some change in notation. As 
an example of another physical mechanism we can consider the system 
coupled to a heat bath at constant temperature f3~^ where some param- 
eters (e.g. interaction coefficients) in the interaction of the components 
of the system are changed. This means that the effective Hamiltonian 
H{t) = H{X{r),y),T G [0,t], of the system is time-dependent with fi- 
nal value Hf{y) = H{X(t),y) and initial value Hi{y) = H{X{0),y). To 
change the parameter A from A(0) to X{t) some heat must fiow from the 
bath into the system so that the change of entropy of the bath equals 
ASb = —P[Hf{yn) — Hiiyo) — Wt\ where Wt is the work done over the 
time [0, t\. If we assume that the initial distribution fi^ = exp[— 
and the final distribution fi^ = exp[—PHf]/Zf are describing equilib- 
rium with respect to the Hamiltonians and Hf respectively, then 
( |5.11| ) becomes 

Rp'ioo) = m{oo) - I3AF (5.12) 

where AF = —(3^^ [In Zf— In Zj\ is the change of (equilibrium) Helmholtz 
free energy. 

This and the previous remark also indicate that the physical signifi- 
cance of the terms — ln/i°(ci;„) -|- ln/i°(a;o) in ( |5.11|) depends on what 
can physically be assumed or said about fi^ and fi^. This can be dif- 
ferent from case to case. Yet here again, as already said in Remark 2 
of the previous section, while these terms have a priori nothing to do 
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with entropy production, adding them gives rise to a more convenient 
form, both for the properties of the average (mean entropy production) 
and for the fluctuations of the entropy production. 

The mean entropy production rate in the steady state where fi^ = 
/i° = /i is time-invariant is obtained from taking the average of ( p.ll| ) 



with respect to P/io^^i. Let Efi stand for the expectation (we do not 
indicate the dependence on /t^). As is the case for our example, we 
suppose for simplicity for the rest of this section that fi^ = /t^vr. We 
have then the 

Steady state analogue of Proposition |4.2| : The entropy produc- 
tion ASb of ( [5. 91 ) satisfies 

E,[e-^^'-] = E.le-^'-^'^'-'^^] (5.13) 

for all complex numbers z. In particular, its expectation equals the 
mean entropy current = mean entropy production = 

E-^ASb] = Y.^kEf.lAE''] > (5.14) 



The relation ( p.l3|) expresses a symmetry in the fluctuations of ASb- 



Modulo some technicalities that amount to estimating space-time bound- 



ary terms, as explained in [16, 18], it reproduces almost immediately 



the Gallavotti-Cohen symmetry, [0]. While it is the theory of smooth 
dynamical systems that has guided us to it, in our analysis, nothing 
has remained of a chaoticity hypothesis. 

The relation ( |5.14| ) states the positivity of the mean entropy produc- 
tion. From its proof (below) we can understand under what (nonequi- 
librium) conditions, it is in fact strictly positive. 

The basic identity that drives fluctuation-symmetry relations is 

£;^[e-^^(^V(^)] = ^;/i[e-(^-")^('^V(0^)] (5.15) 
for every function ip of the trajectory of the system and with 



This identity ( |5.15|) follows from the very definition of TZ as the log- 



arithmic ratio of two probabilities from which also lZ{Quj) = —lZ{uj) 
The equation (|5.13|) follows simply by taking for ip in (|5.15|) , 

[/i(a;o)//i(7ru;„)]' 



Before we give the proof of ( ^.14 ), we give the version for the transient 



regime of the system (steady state for the reservoirs): 
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Transient regime analogue of Proposition |4.2| : Recall the nota- 
tion (|5.1CI| ). Then, 



,r, i e 



Its expectation equals 



(5.16) 



+ ^/3fc£;^o^^i[AE'^] >0 (5.17) 



The relation ( |5.16| ) for the example ( p. 121 ) gives the irreversible work 
- free energy relation of Jarzynski, |p. 

Now to the proofs of ( p3^ )-(|^T7|). As before in (|]3p, ( pTTBD just ex- 



presses a normalization of the probability measure Pf^o^^^iQ. The 



equality in (|5.17|) follows as in (|4.6| ) from taking the expectation of 
(|5.11| ). From the Jensen inequality applied to (|5.16|) , we obtain the 
inequality in (|5.17|) . The relation (|5.14|) now follows from applying sta- 
tionarity = fit = Ji. 

We thus see that the positivity in ( p.l4| ) and in ( ^.17| ) follows from con- 
vexity. By the same argument, the strict positivity will express that 
the two path space measures P^o^^i and P^o^^iG are really different, 
i.e., applying time-reversal really has an effect. (In this is expressed 
via the relative entropy between these two path space measures.) 
Remark 3: Note that the above fluctuation identities (|5.13|) , ( ^.15|) 
and ( |5.16| ) do not depend on Assumption A2. We can repeat them 
directly starting from 



6. Markov approximation 

The stochastic processes of the previous sections give the statistics of 
trajectories for reduced states induced by the Hamiltonian dynamics. 
The stochasticity does not represent microscopic or intrinsic random- 
ness, whatever that means, and is not an easy substitute for chaoticity. 
In the present section we make an approximation for this stochastic 
evolution that does go in the direction of assuming some chaoticity but 
again on the level of reduced states. 

6.1. Closed systems. We refer here to Section^. Look at the time- 
evolved measure (/i x p)t starting from fi x p at time zero: for t = n6, 

(A X P)tix) = ftx p(/-"x) 

Remember that we have used before its projection fit on T. Observe 
now that, quite generally, {fi x p)t ^ fit ^ P- That is: the phase space 
distribution does not remain microcanonical; when two points x, ?/ G F 
fall into the same reduced state (Mx = My), it need not be that 
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{flxp)t{x) = {fixp)t{y). This is an instance of so called memory-effects; 
the process does certainly not correspond to a Markov process on 
f. 

We can obtain a Markovian approximation by forcing uniformization 
at each step in the evolution. We then define the discrete time Markov 
approximation via the updating 

fin = Piifj'n-i y< p)s), n = 1,2, .. . (6.1) 

or more explicitly, from (|3.1| ), 

~ ~ ^n.>^\f'^M nM'\ 



M'er 



corresponding to the Markov chain on F with transition probabilities 
p{M',M) = If^^M n M'\/\M'\. Naturally it satisfies the detailed bal- 
ance condition 

|M| p{M, M') = \M'\p{ttM', ttM) (6.2) 



or 



PiM,M') ^ SsiM')-SMM) (Q 3^ 

p{nM',7iM) ^ ■ ' 



It is an approximation in the sense that the evolution defined by ( |6.1|) 
corresponds to a repeated randomization of the 'true' evolution. We 
expect it to be a good approximation in so far that \M fl f^M'\ ~ 
|M||M'|/|r|. That is to say, for 6 large enough for the averaging over 
the reduced state to be valid. That is a mixing condition but for 
the evolution over the reduced states (as for Gibbs' inkdrop), see 
for similar remarks. It also implies relaxation to equilibrium. Usu- 
ally however this is combined with other limiting procedures through 
which the reduced variables (or their fluctuations) get an autonomous 
(stochastic) evolution. Most important in all this however remains the 
'proper choice' of reduced states (or, thermodynamic variables). 

We now have a Markov chain (X^) on T with transition probabilities 
p{M, M') and 

Prob^[X„ = M„, . . . , Xo = Mo] = p{Mo)p{Mo, Mi) . . . p(M„_i, M„) 

is the probability of a trajectory ui = (Mq, Mi, . . . , Mn) G r"+^ when 
the Markov chain was started from the probability measure jl. We have 
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instead of (U): 

Prob^(X„ = M„, . . ■ ,Xo = Mo) 
Prob^^(X„ = vrMo, . . . , Xq = vrMj 



Upon substituting (|6.2|), the exponential in ( |6.4| ) equals |M„|/|Mo| and, 
perhaps surprisingly, the identity ( [4 .21 ) -( ^31) is unaffected in the Markov 
approximation: 

Prob^(X„ = M^, ■ ■ ■ ,Xo = Mo) ^ /2(Mo) |M^| 

Probi>,(X„ = ttMo, . . . , Xo = 7rM„) z>(M„) |Mo| 

Furthermore, take now 9 = fin oi ( |6.1D and let us denote as in Propo- 



sition 4.2 



. ^ Prob^(X„ = M„, ■ ■ ■ ,Xo = Mo) 
'^^"'^ " Prob^„^(X„ = ttMo, . . . , Xo = 7rM„) 

Its expectation, as in ( [4 .61 ), under the now Markovian path space mea- 
sure is 

E^m]= J2 p-MRi{^) = s{m-s{iin\p)>^ (6.6) 

the difference of relative entropies with respect to the stationary (re- 
versible) probability measure p{M) = |M|/|r|; the relative entropy is 
defined from S{i>\p) = Y.M^W\nV{M) / p{M). The identities 
and ( [4 .61 ) are consistent since the Gibbs entropy can be written in terms 
of this relative entropy as Sg{i') = In |r| — S{y\p). 

6.2. Open systems. We refer here to Section |^. In the same spirit 
as above, we get the Markov approximation for open systems by fol- 
lowing the procedure of Section ^. We now get Markov processes with 
transition probabilities 

q{M,M') = PM^f.AM.M') 

where we have understood pP = 5(M — •). We will again suppose for the 
enviroment that p^Tc = p^. These transition probabilities then satisfy, 
from (|5.11 ), 

The measures p of above now correspond to the distribution of the 
internal degrees of freedom (the open system). The important change 
is that detailed balance may be violated from the action of reservoirs 
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maintained at different but fixed temperatures or chemical potentials. 
We can for example substitute ( |5.9| ) in ( |6.7|) to retain only a local 
detailed balance condition, that is 



q{M, M') 
q{nM',nM) 



exp[5°(M') - SUM) + Y,Pk^E\M,M')] 



Depending on the transition M — > M', in particular, where this tran- 
sition of the state of the system is localized, various terms in the expo- 
nential can become zero or non-zero, see also ( |6.14| ) below. 



While the formal structure of the Markov approximation for open sys- 
tems runs exactly similar to what we did for closed systems, cf. ( |6.1| ), 
we remark that its validity now requires more than what was mentioned 
following ( [6.31 ). In fact, a competing requirement enters if we wish to 
maintain assumption A2 of the previous section. Assumption A2 will 
be more reliable in so far as the 5 (i.e., the time steps in the trajectory 
of reduced states) is smaller while the mixing condition on the level of 
reduced states that justifies the Markov approximation requires large 
enough 5. Again, as mentioned following assumption A2, this moti- 
vates using different time scales for the evolution of the reduced states 
in system and enviroment. 

We now have a Markov chain (X/^) on CP with transition probabilities 
q{M,M'), and 

Prob^[X„ = M„, . . . , Xo = Mo] = jl{Mo)q{Mo, M{) . . . g(M„_i, M„) 

is the probability of a trajectory oj = {Mq, Mi, . . . ,Mn) when the 
Markov chain was started from the probability measure fl, we have 
instead of (O): 



Prob^(X„ = M„, . . . ,Xo = Mo) 
Prob^^(X„ = ttMo, . . . , Xo = vrM,) 



z>(M„) """"^'f-^ g(7rMfc+i, vrMfe 



El qy^vik,^y-ik+i) 1 , 



k=0 



As motivated in Section ^, its logarithm will continue to interest us as 
variable entropy production. 



From ( |6.8| ), we see that the variable entropy production is now given 
by: 

y^^AMhi^^ (6 9) 

V g(vrMfc+i,7rMfc) 
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Furthermore, for open systems, the relation (|6.6| ) gets replaced with 
E-,[Ry\ = (6.10) 

n— 1 

S{~^^) - S{~^) + MM)g{M, M') In > 

k=0 M,M' ' 

and there is in general no way to write this as a change in relative 
entropies S{fl\p) — S{p,n\p)- In other words, in general, there is no 
role for the time-derivative of the relative Shannon entropy as total 
entropy production. When fl = fin is stationary for the Markov chain, 
the right-hand side of the equality in (|6.10|) gives us the mean entropy 
production rate as 

M,M'ef 

which (up to the inclusion of the time-reversal involution vr) is the 
standard expression for an effective Markovian dynamics modeling a 
nonequilibrium steady state, see e.g. If. Note that if fl is stationary 
under updating with transition probabilities q{M,M'), then ftTc is sta- 
tionary under updating with the transition probabilities Gg(M, M') = 
q{7cM',7TM)fL{'n'M')/fi{iTM) for the time-reversed process. It is then 
easy to see that the mean entropy production rate is positive and 
equal for both stationary processes. Or, the mean entropy production 
is time- reversal invariant. This again is ultimately a consequence of 
the dynamic reversibility of the microscopic dynamics and it yields in- 
teresting by-products (like Onsager reciprocities) as discussed in |T^. 
For the pathwise expression of the entropy production rate, we look 
back at (|6.9|). The entropy production per time-step is 

g(7ru;fc+i,7rc^fc) 

Note again that a^{Qu) = —a''^{uj) and that, when /i is stationary, 
i?»/n=iln^ + aS(..) 



This leads again as in (|5.13|) and in ( |5.15|) almost directly to a Gallavotti- 
Cohen symmetry, 0, |T^ . 

For a continuous time Markov chain (Xt) on a finite set with tran- 
sition rates k{M,M'), similarly, the entropy production rate in the 
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distribution fi is given by 

. 1 Y:MM. M'MM) - KM'. M).(M')1 in 

(6.13) 

(We have set tt = identity for simplicity.) Let us take 

k{M,M') = A;o(M,M')e"'^(*^'^'^')/2 

where ko is the rate for a detailed balance evolution with unique re- 
versible measure fiQ. We assume that there is a unique stationary 
measure /Xg with e measuring the distance from equilibrium: 

Here ip'^^M.M') = -ip'^^M'.M) = [ip{M,M') - V^(M',M)]/2 origi- 
nates in some driving. We did not indicate it but the k{M, M') and 
therefore the functional a in (|6.13|) depend now on e. One can then 
check that is minimal for a probability measure /i* which coincides 
with /ig to first order in e (minimum entropy production principle). A 
special case of this calculation can be found in 0. We give the general 
statement and argument in Appendix C. 

We next apply the above scheme for a Markov approximation for 
closed systems to a diffusion process that appeared in the Onsager- 
Machlup paper, 



7. Application: Gaussian fluctuations 

As we have argued before, the entropy production appears as the 
source term of time-reversal breaking in the logarithm of the prob- 
ability for a preassigned succession of thermodynamic states. Such 
calculations were already done to study the fluctuations in irreversible 
processes in the work of Onsager and Machlup in 1953, |T^. Our pre- 



vious section is some extension of this, as we will now indicate. 



We only redo the very simplest case of [0, their Section 4 for a single 
thermodynamic variable a obeying the equation (in their notation) 

Ra + sa = e (7.1) 

We do not explain here the origin of this equation except for mention- 
ing that R relates the thermodynamic force to the flux a (assumption 
of linearity). The constant s finds its origin in an expansion of the 
thermodynamic entropy function Sxia) = St{0) — so? 12 around equi- 
librium. For every a, St[pi) is the equilibrium entropy when the system 
is constrained to this macroscopic value and can be identified with the 
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Boltzmann entropy 83(0:) (up to the thermodjTiamic limit) which is 
also defined outside equilibrium. From the expansion of the entropy, 
the thermodynamic force dSx/da depends linearly on the variable a 
(Gaussian fiuctuations). The right-hand side of ( [7.1|) is purely ran- 
dom (white noise) with variance 2R. In this way the oscillator process 
da = —s/R adt + ^J2/R dWt with Wt a standard Wiener process, is 
obtained for the variable a. 

The work in is then to calculate the probability of 'any path.' 
These are the trajectories we had before. With the current methods of 
stochastic calculus, this is not so difficult. 

We proceed with ( |7. 1| ) . Using the path-integral formalism we can write 
the 'probability' of any path uj = (a;(r),r G [0,t]) with respect to the 
fiat path space 'measure' du = [da{T)]: 

Prob^(u;) ~/2(a(0))e~^(") 

for some initial distribution fi and with action functional 
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A{uj) = ^ I R{a{T)+-fa{T)fdT (7.2) 
4 Jo 

for = s/R. There is no problem to make mathematical sense of this; 
for example the cross-product 

t j-t 

aadr = a o da 



Jo 

is really a Stratonovich integral and the exponent of the square can 
be combined with the fiat path space measure to define the Brownian 
reference measure. More to the point here is that the integrand in the 
action functional A can be rewritten as 

Ra\T) + ^a^{T) + -^{sa\T)) 
R dT 

The last term is minus twice the variable entropy production rate 
St{c()- It is the only term in the integrand that is odd under time- 
reversal. So if we take the ratio as in ( [4.4|) but here with 71 = identity, 
we get, rigorously, 

dP- 

ln^^(a;) = [Sriait)) - 5^(^(0))] + [- ln/i,(a(t)) + ln/i(a(0))] 

(7.3) 

so that, just as in ( [4.4|) , indeed the change in thermodynamic entropy 
is obtained from the source term in the action functional that breaks 
the time-reversal invariance. 

Onsager and Machlup use the expression ( |7.2| ) for the action functional 
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to derive a variational principle that extends the so called Rayleigh 
Principle of Least Dissipation. The idea is to take t very small and 
to seek the a{t) which will maximize the probability Prob[a;(t)|a(0)]. 
Or, what is the most probable value of the flux a when you start from 
a(0)? This then determines the most probable path. This means that 
we should maximize 

over all possible a, or that we should take 

St — = max. 

where ^(d) = Ra'^/2 is the so called dissipation function. In other 
words, the maximum (over the flux) of the difference of the entropy 
production rate and the dissipation function determines the most prob- 
able path given an initial value for the thermodynamic variable. We 
mention this here not only because it is a central topic in the Onsager- 
Machlup paper but because this Rayleigh principle is often confused 
with the minimum entropy production principle that we had at the end 
of Section p. In fact, the Rayleigh principle is more like a maximum 
entropy production principle (similar to the Gibbs variational principle 
in equilibrium statistical mechanics) enabling the search for the typ- 
ical histories. Of course, its solution is just ( [7. 1| ) for e = 0, i.e., the 
deterministic evolution for the thermodynamic variable, cf. [0]. The 



minimum entropy production principle on the other hand, attempts to 
characterize the stationary states as those where the entropy produc- 
tion rate is minimal. Both principles have serious limitations. 



8. Phase space contraction 

A more recent attempt to model nonequilibrium phenomena that 
was largely motivated by concerns of simulation and numerical work, 
involves so called thermostated dynamics, see [§, These are again 
as in the previous section, effective models but now using a determinis- 
tic dynamics. First, non-Hamiltonian external forces are added to the 
original Hamiltonian equations of motion to keep the system outside 
equilibrium. Since then, energy is no longer conserved and the system 
would escape the compact surface of constant energy, one adds 'ther- 
mostat forces', maintaining the energy fixed. There are other possible 
choices but they do not matter here. The resulting dynamics no longer 
preserves the phase space volume. We will keep the same notation as 
in Section |^ to denote the discretized dynamics; / is still an invertible 
transformation on P satisfying dynamic reversibility tt/tt = /^^ but 
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now the Liouville measure is not left invariant. It is important to re- 
member that r does no longer represent the phase space of the total 
system (subsystem plus reservoirs); it is the phase space of the subsys- 
tem while the action of the environment is effectively incorporated in 
/. This environment has two functions at once: it drives the subsys- 
tem in a nonequilibrium state and it consists of a reservoir in which all 
dissipated heat can leak. 

In the same context it has been repeatedly argued that the phase space 
contraction plays the role of entropy production, see e.g. |^ |^. For 
thermostated dynamics, there are indeed good reasons to identify the 
two and various examples, mostly applied in numerical work, have 
illustrated this. Yet, from a more fundamental point of view, this 
needs an argument. To start, there is the simple observation that en- 
tropy can change in closed Hamiltonian systems while there is no phase 
space contraction. Moreover, even when used for open systems in the 
steady state regime, entropy production as commonly understood in 
irreversible thermodynamics is more than a purely dynamical concept. 
It is also a statistical object connecting the microscopic complexity 
with macroscopic behavior. That was also the reason to introduce the 
reduced states and the partitions T,Q. It is therefore interesting to 
see how and when phase space contraction relates to the concept of 
entropy production that we have introduced before. 

Since the set-up is here somewhat different from that of Section |^, 
we denote here the state space by M. instead of by T. It need not be 
the set of microstates (as in thermostated dynamics); it may be the 
set of possible values for some hydrodynamic variables, more like our 
set r. We think of as a bounded closed and smooth region of M''. 
Still, the dynamics / is assumed dynamically reversible (which would 
fail for irreversible hydrodynamics). 

Suppose we have probability densities /i and u on Ai. We replay ( [4.2| ) 
or ( |5.4| ) but now on the space A4. For every function on A4^~^^, let 
be given as $* (xq, Xi, . . . , x„) = $n(7rx„, nXn-i, ... , tixq). We find 
that 
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using dynamic reversibility. Now change variables y = f^x, 
$* (x, fx, . . . , f'^x)vTx{x)dx = 

/ ... — --—\ — {x)\fi{x)dx 

J jj[x) dx 



or 



ui f^r) df^ 



fi{x) dx 

This should again be compared with ( |4.2| ) and with ( |5.4|) . In particular, 
we see that the phase space contraction, or more precisely, minus the 
logarithm of the Jacobian determinant — In {df^/dxl, replaces the total 
entropy production (of the total system) we had before: 

SBiFx) - Sb{x) - In \dr/dx\ (8.1) 

This requires the dynamical reversibility; without it, even this purely 
formal identification is not justified. 



Looking further to compare with Proposition [4.1| and ( |5.8|) , we can take 
z/(x) = /i„(x), the time-evolved density. Then, 

u{rx) = ix{x)\^{rx)l r^{x) = l 
dx 

so that the formal analogue of the right-hand side of (|4.4| ) and ( |5.8|) 
now becomes 

-ln/i„(/"x) +ln/i(x) -ln|^(x)| = (8.2) 

dx 

But if we believe in our algorithm for computing the mean entropy 
production as in ( [4.6| ) for closed systems and as in ( |5.17| ) for open 



systems, the expectation of ( p. 21 ) with respect to /x should give us the 
mean entropy production; it remains of course zero: 

f f f df"' 

— / dxjj,n{x)\n jj,n{x) + / dxjj,{x)\n jj,{x) — / dxjj,{x)\n —^{x) = 

(8.3) 

In other words, we find that the mean entropy production is zero. 
Heuristically, this is quite natural by the very philosophy of the ther- 
mostated dynamics; the change of entropy in the subsystem is exactly 
canceled by the change of entropy in the environment. That is: the 
difference in Shannon entropies is given by the expected phase space 
contraction. This is known since at least 

It is true that the above and in particular (|8.2|) concerns the tran- 
sient regime and that the above calculation cannot be repeated for the 



30 



stationary measure as it may be singular. Yet, this property may be 
considered as an artifact of the infinitely fine resolution in Ai and we 
can remove it by taking a finite partition of A^. We need a gen- 
eralization of ( ^.2| ) for dynamics that do not preserve the phase space 
volume, pf~^ 7^ p, with p{dx) = dx the flat measure on M.. Using the 
notation of Section ^ we now get 



Prob^ e M„,... ,xo e Mo) 
Prob/i^^xp(a;n e ttMo, . . . , G 7rM„) 

/i(Mo) p(M„) pinj^of-'M, 



Pn{Mn) p{Mo) pfHr^1=J-^M,) 



by using again the dynamic reversibility of the map /. In the stationary 
regime, the formal analogue of the entropy production rate equals 

1 , Prob^xp(a;n e M„, . . . , Xq G Mq) _ 



lim — In 

n n Prob^„^xp(x„ G vrMo, ... , Xq G 7rM„) 

1 p(n^„/-^M,) 

" n p/»(nj,„/-^A/,) ' 

Note that while this is true for every finite partition Ai, it fails for 
the finest partition where Ai would coincide with the original phase 
space A4. The above formula may be further elaborated, assuming that 
the partition Ai is generating for /. (This would not be true for the 
partition T corresponding to the physical coarse-graining induced by a 
set of thermodynamic variables). Let x & Aihe fixed and choose Mj = 
M{px). Using the notation M^"^ = n'J^J-^Mj, we have Mi"+^^ C 

Mi*"^ and n„Mi"^ = {x}. Suppose now that the following limits are 
equal: 

Clearly, {dpf /dp){x) is a general form of the phase space contraction 
(the Jacobian determinant of /) . The right-hand side takes its ergodic 
average. If we sample x & A4 from the fiat measure p, we could sup- 
pose that these ergodic averages converge to the expected phase space 
contraction for some distribution p on At. That would for example be 
guaranteed under some chaoticity assumptions for the dynamics /; in 



particular if the dynamical system allows a SRB state p, |20|. We can 



then combine the previous two relations and find that, for p-almost 
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every x G Ai, the mean entropy production rate gets the form 

hm ^ In P^Q^/'xp(^" ^ Mjf'^x), ■ ■ ■ , Xq G M(x)) ^ ^ cip \ 

Tn ""Prob^^^xpvrla^n e 7rM(a;),... ,a;o e 7rM(/"a;)) ''v''ci(p/)y 

(8.6) 

This is exactly the mean entropy production rate one works with in 
thermostated dynamics, see e.g. [^]. Comparing it with ( [4.6| ) and 
(|5A^ )- (|5T7 | ) , it does indeed repl ace the mean entropy production as 
computed from the algorithms in Sections ^ and ^. 
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Appendix A. Kac ring model 

The scheme of Section ^ can also be applied to every model dynam- 
ics sharing the property of dynamical reversibility with Hamiltonian 
dynamics. To illustrate this and in order to specify some quantities 
that have appeared above, we briefly discuss the so called Kac ring 
model. We refer to the original |]12| for the context and to 0] for more 



discussion. 

The microscopic state space is = { — Its elements are de- 
noted by a; = f ) = (?7i, . . . v) and the kinematic time- reversal 
is TTx = (?7i,... ,riN;—v). The microscopic dynamics fs depends on 
parameters Ei = = 1, . . . , N, and is defined as 

feiVl, ■■■ iVn\ +1) = i^NVN, SlVl, ■■■ , ^N^iVN-I, +1) 
feiVl, ■■■ ,Vn; -1) = iElV2, £2V3, • • • , ^NVU -1) 

so that /e = vr/^^^TT (dynamic reversibility). The only information 
about the parameters is that Yli = ""^^ some fixed m. 
Since the "velocity" v is conserved, we can as well study the dynam- 
ics on r = {— (fixing v = +1) and to each microstate r] 
we associate the macroscopic variable a{r]) = YliiVil^- This intro- 
duces the partition f containing + 1 elements. For example, the 
set M{ri) G F contains all the cr G F for which a{ri) = a{a) and 
|M(?7)| = CAr((a(?7) + l)N/2) the binomial factor. Trajectories can 
therefore be identified with a sequence of macroscopic values aj. We 
are interested in the case of finite (but possibly long) trajectories while 
taking N extremely large. In the simplest approximation, this means 
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that we let — > +00. It can be shown that for the overwhelming ma- 
jority of the Ei, the macroscopic value an after n time steps behaves as 
an = m^tto with ao the initial macro-value. The limiting evolution on 
the level of macrostates is therefore deterministic but not time-reversal 
invariant. Equilibrium corresponds to a = 0. The entropy production 
rate (per degree of freedom) when the system has macro-value a is 
(1 + a) In y/1 + a + (1 — a) In \/l — a — (1 + ma) In \/l + ma — (1 — 
ma) ln^/l — ma = (1 — vp?)cP' j1 up to second order in a . 

Various examples of applying exactly the algorithm of Section ^ to 
compute the entropy production and to study its fluctuations have 
appeared before, see [l^, [l^, We just add here how the Markov 
approximation for the Kac ring model looks like. 
The transition probability can be read from ( |6.1|) : for finite A^, a') 
is the probability that the macroscopic value equals a' after one time 
step, when the process was started from a randomly chosen 77 with 
macroscopic value 'YliiVi/^ = 

p{a,a') = c^(^(^^li^j^^/2) ^^^ ^ ^ ' 5^^' " ^"^^ ^^iVi = "'^}| 

Depending on the parameters Ei, this will often be zero, certainly when 
is large and a' is far from equal to ma. On the other hand, when 
a' = ma ± y/l — m^ / a/TV, the transition will be possible but damped 
as exp[— A^(a' — ma)^/2(l — m^)]. It is therefore interesting to study 
the evolution on the level of the rescaled variables y/Na; these are the 
fluctuations. This takes us back to Section |^. In equation (|7.1|) , we 
should take i? = 1/(1 — m) and s = 1. The solution of the Rayleigh 
principle is of course here found from maximizing the transition prob- 
ability p{a, a') and this happens when a' = ma. As always with this 
principle, see [jll|, this does not teach us anything new; it only gives a 
variational characterization of the hydrodynamic evolution. 

Appendix B. Hamiltonian dynamics of composed systems 

In order to clear up the content of assumptions Al and A2 of Section 
^ we demonstrate here how it naturally emerges in the framework of 
Hamiltonian dynamics. We again have in mind a composed system 
consisting of a system thermally coupled to an environment, the latter 
having the form of a few subsystems (reservoirs). 
Let T be a finite set whose elements label the individual particles of the 
total system. That means that we are really considering a solid (and 
not a fluid). To every particle i G T, there is associated a position 
and momentum variable Xj = {qi,Pi). Given a configuration x, we put 



33 



xa = (^AjPa) for the coordinates of particles befonging to the system 
A C T. We thus decompose the set of particles T by splitting the total 
system into a system and m reservoirs, T = A U U . . . U V"^. We 
assume that the Hamiltonian of the total system may be written in the 
form H{x) = H^xa) + ELi H^ilv^^Pv^) where V^ = V^\J dkA and 
9fcA C A is the set of all particles of the system coupled to the k—ih 
reservoir. Moreover, we need the assumption that the reservoirs are 
mutually separated in the sense dkA fl df A = whenever k ^ £. To be 
specific, consider the following form of the Hamiltonian: 

ieA * (y)cA 

2 



(B.2) 



For what follows, we consider another decomposition of the energy 
of the system in the form H^{xa) = h^ilK^Php) + Efc=i h^{xd^A) where 
A° = A \ Ufc9fcA. We can take, for instance, 

2 

h\qK.PK-) = E + ^^(^^)] + E ^(^^ - ^^■) (^-3) 
h\xa,A) - E + ^^(^0] (B.4) 

If the trajectory a;(r) = (5(t),p(t)) is a solution of the Hamiltonian 
equations of motion, then the time-derivative of the energy of each 
reservoir is in terms of Poisson brackets: 

^((^(r)) = {H\ H}{u{r)) = {H\ H'}{u{r)) ^^^^^ 
^{H^h'}{u;{T)) 
A similar calculation yields 



u;{T))^{h\H}{u;{r)) 



dT^ ^ " I ' V y/ (B_g) 

^{h\H'}{u{T)) + {h\H'^}{u{T)) 



where in the last equality we used the assumption that the reservoirs are 
mutually separated. Combining the above equations and integrating 
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^\q^{r) - q,{r)) (B.7) 



Notice that the right-hand side depends only on the restriction ut^it) of 
the trajectory uo. Therefore, assuming that co'(t), t^'(t) are two solutions 
of the equations of motion such that uj{^{r) = uj'j^{t), t G (tQ,t), the 
heat flow into the A;— th reservoir, Q\tQt){'-o) = H^{ijj{t)) — H''{uj{to)) 

satisfies Q^tot)(^) ~ Q{tot)(^')- -^^^ differently, the heat current into 
each reservoir, being a state quantity from the point of view of the total 
system, is also a functional of the (complete) trajectory of the system. 
This motivates assumption A2. Moreover, from the above calculation, 
also the assumption Al becomes plausible as we can expect that the 
right-hand side of ( p.7| ) is of order {t — to)\dkA\. 

Remark: Note that the decomposition of the total energy into local 
parts is not unique due to the presence of interaction. The above 
claim is only true for the decomposition in which the interaction energy 
between the system end each reservoir is taken as part of the energy 
of the reservoir. However, the difference between this reservoir energy 
and others can only be of the order of \dkA\ which is again sufficient. 
Furthermore, all possible decompositions become undistinguishable in 
the regime of weak coupling. 

Appendix C. Minimum entropy production principle 

In this appendix we examine the validity of the minimum entropy 
production principle in case of Markov chains breaking the detailed 
balance condition, as promised at the end of Section p.. We use the 
same notation as there, namely we consider a continuous time Markov 
chain (Xt) on a finite state space with transition rates ke{M, M'). The 
latter are parameterized by e measuring the distance from equilibrium. 
More precisely, let ^^(M, M') = ko{M, M') exp[£?/'(M, M')/2] where the 
Markov chain with rates kQ{M, M') has a unique reversible measure /io, 
i.e. , 



We also assume that iio{M) ^ for all M. The stationary measure [i^ 
is a solution of the equation 



^io{M)ko{M, M') = /io(M')A;o(M', M) 



(C.l) 



Y,[^^e{M)k,{M,M') - ii,{M')k,{M' ,M)] = 



(C.2) 



M 
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for all M' . Write this measure in the form = + ef + o{£)) 
with the normalization condition m f {M) = 0. Then a simple 

calculation yields the following (linearized) equation for stationarity: 



^/io(M)fco(M,M')[/(M) - f{M')+ij^{M,M')] = (C.3) 



where we used il)^{M, M') to denote the asymmetrical part of the driv- 
ing, V''''(M,M') = [i){M,M') -ip{M',M)]/2. This equation with the 
constraint ^jvf /^o(^)/(^) = has always a solution, we will assume 
it is unique. Notice that, up to first order in e, only the asymmetric 
part of the driving deforms the stationary measure. 

We now compare this result with that of the minimum entropy pro- 
duction principle. Recall that the entropy production rate is the func- 
tional on measures 

The first observation is that it is convex. So, the constrained variational 
problem ^"^(/i*) = min, ^a.//U*(M) = 1, is equivalent to solving the 
equation 



together with J2m = 1- We again linearize this equation by 

writing /i* = /io(l + ef* + o{e)) and after some calculation we get 



5^/io(M)A;o(M, M')[r(M) - r(M') + ^P^{M, M') - A] = 



if the minimizing point (i* is unique, it must correspond to /* = / with 
/ being the normalized solution of ( |C3D . 

Note that in higher orders the minimum entropy production princi- 
ple fails as a variational principle for the stationary measure. But even 
to linear order, outside the context of Markov processes, the principle 
can be questioned both for its correctness and for its usefulness, see 



Appendix D. Systems in local thermodynamic equilibrium 



M 




(C.5) 




111. 



In this appendix we connect our presentation of Section ^ with the 
standard formulations of irreversible thermodynamics. We go about 
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this in a rather formal way, trying to safeguard the simphcity of the 
explanation. 

We consider the system itself to be large (yet small when compared 
with the reservoirs) and we split it further into (still large) subsystems 
around a spatial point r. We assume that these subsystems are in 
(local) equilibrium so that the change of entropy S^{uJn) — S'^{^^o) of the 
system (appearing in (pD or (pD) is a change of (maximal) Boltzmann 
entropy when the energy in the subsystem around r moves from the 
value U{r,0) to U{r,t). That is, 

SU^n) - SU^o) = J^^Snir, f/(r, t)) - ^^(r, [/(r, 0))] 

r 

where 5*5 (r, U) is the logarithm of the phase space volume of the sub- 
system around r corresponding to energy value U. We start from {f).9\). 
It gets the form 

ASBico) = J2[SBir,Uir,t))-SBir,Uir,0))] + J2Pk^E'^i^) (D-1) 

r k 

Here the first sum runs over the subsystems of the system under consid- 
eration, while the second sum is taken over all reservoirs. We introduce 
the temperature of the r-subsystem as /5(r, r) = {dSB/dU){r, U{r, r)), 
and then 




We use J(r, r', r) to denote the energy current at time r from the r- 
subsystem to the r'-subsystem. Similarly, J^{r, r) stands for the energy 
current from the r-subsystem to the A;— th reservoir. The conservation 
of energy then implies the equalities 

y k 

and 

AE''{uj) = y2[ dr j''{r,T) (D.4) 

The currents are antisymmetric: J(r, r', r) = —J{r', r, r). The entropy 
production now becomes 

A5b(c.) = r dr [J2 Y.(f^k - /3(r, r))j\T, r) + /3(r, r) J(r', r, r)] 

^ k r r,r' 

(D.5) 
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The first term on the right is a surface sum. Its origin is the entropy 
current. We assume that every subsystem is coupled to at most one 
reservoir. In the continuum, if r is at the boundary of the system with 
the A;— th reservoir, then in fact /9(r, r) = j3k. Hence, for the first term, 
either J^{r,T) = or (3{r,T) = /3k which makes it vanish. When we 
are dealing with closed systems, then J'^(r, r) = by definition. Using 
further the antisymmetry of the bulk currents, we obtain 




drY,^P{J){T,r) (D.6) 



where we used the notation 

Vm(r,r)^Y. ^'^'''^'^-^^''^h (r,r\r) (D.7) 

r' 

This is already close to the standard formulations in which the entropy 
production rate equals a thermodynamic force times a current. Indeed, 
assuming that the decomposition of the system into subsystems has a 
natural space structure, say as the regular Z'^-lattice, and that the cur- 
rent exchanges take place only between neighboring subsystems (via 
their common interface), we can write V/?( J)(r, r) ~ V/3(r, r) • J(r, r) 
(the derivative taken in the discrete sense). The (total) entropy pro- 
duction is then A5'b((x)) = Jl^'^Ylir^i'^^'^) with space-time entropy 
production rate 

(7(r, r) = V/3(r, r) • J(r, r) 

as sought. 
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